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A simulation study is presented to evaluate and compare three methods to 
estimate the variance of the estimates of the parameters <3 and C of the signal 
detection theory (SDT). Several methods have been proposed to calculate 
the variance of their estimators, d' and c. Those methods have been mostly 
assessed by comparing the empirical means and variances in simulation 
studies with the calculations done with the parametric values of the 
probabilities of giving a yes response on a signal trial (hits) and on a noise 
trial (false alarms). In practical contexts the variance must be estimated from 
estimations of those probabilities (empirical rates of hits and false alarms). 
The three methods to estimate the variance compared in the present 
simulation study are based in the binomial distribution of Miller, the normal 
approach of Gourevitch and Galanter and the maximum likelihood method 
proposed by Dorfman and Alf. They are compared in terms of relative bias 
(accuracy) and the mean squared error (precision). The results show that the 
last two methods behave indistinguishably for practical purposes and 
provide severe over-estimation errors in a range of situations that while not 
the most common are perfectly credible in several practical contexts. By 
contrast, the method of Miller provides better results (or at least similar) in 
all conditions studied. It is the recommended method to obtain estimates of 
the variances of these statistics for practical purposes. 
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Sometimes we are interested in knowing the variance of the estimates 
of the parameters of the signal detection theory (SDT; Green & Swets, 
1966; MacMillan & Creelman, 2005; Wickens, 2001). For example, for 
testing hypotheses about those parameters (Jesteadt, 2005; Miller, 1996; 
Verde, MacMillan, & Rotello, 2006) or when we want to perform a meta- 
analytic integration of the evidence on a specific issue and the studies have 
been conducted analyzing statistics associated with the SDT. Specifically, 
the indices d' and c, estimators of d and C, which are measures of sensitivity 
and response bias respectively, are often used to reflect the effects sought in 
experimental studies (Logan, 2004; Swets, Dawes, & Monahan, 2000). In a 
meta-analysis the statistics provided by the primary studies are combined to 
yield a point estimate of the effect size. The most common method to 
combine the estimates consists in calculating a weighted average of the 
values, using as weights the reciprocals of their variances (weight = l/o 2 ) 
(Borenstein, Hedges, Higgins, & Rothstein, 2009; Botella & Sanchez-Meca, 
2015; Hedges & Olkin, 1985). Accordingly, to implement such a procedure 
it is necessary to know the variance of these statistics in each primary study. 

Assuming the normal homoscedastic (NH) SDT model, and the 
yes/no experimental paradigms (MacMillan & Creelman, 2005), three main 
methods have been proposed for calculating the variance of d' (see below 
for a detailed technical presentation): the exact method of Miller (1996), the 
approximate method of Gourevitch and Galanter (1967), and the maximum 
likelihood method of Dorfman and Alf (1968). 

The methods of Miller (1996) and Gourevitch and Galanter (1967) 
compute the variance of d’ substituting in their formulas the conditional 
probabilities of a false alarm and a hit. The variance of d’ is properly 
calculated when using those probabilities, and the obtained value is the true 
(parametric) variance. But this only can be done if the true probabilities are 
known, as in simulation studies. In those contexts where the true 
probabilities are unknown the variance of d’ is calculated using the 
proportions of hits and false alarms obtained in a finite number of trials. 
Consequently, because these are estimators of the probabilities, the variance 
is an estimator of the true (parametric) variance. 

Two studies have been conducted in order to compare some of the 
three methods. Miller (1996) calculated the variance of d’ applying his 
procedure and the method of Gourevitch and Galanter. The variance of d’ 
was calculated for different values of d and number of trials with the same 
response bias (unbiased responding). The results show that: a) the variance 
of d’ calculated by Miller’s method and d is increasing: the variance of d’ 
increases to a maximum and then decreases, where the position of this 
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maximum (a d value) depends on the number of trials; b) the variance of d’ 
calculated by the method of of Gourevitch and Galanter and d is increasing: 
the variance of d’ increases. 

In a Monte Carlo study, Kadlec (1999) compares the empirical 
variance of d’ obtained in the simulation with those calculated using the 
method of Gourevitch and Galanter. Three variables were manipulated: d, 
the number of trials and response bias. According to the results of Kadlec 
(Figure 10), the variance obtained by the method of Gourevitch and 
Galanter is similar to the empirical variance until a critical d. Above this 
critical value, the method of Gourevitch and Galanter overestimated the 
variance of d’. The critical d value depends on the number of trials and the 
response bias. 

It is important to mention that in Miller (1996) the variance of d’ was 
calculated using the parametric probabilities of false alarms and hits; on the 
contrary, in Kadlec (1999) the variance of d’ following Gourevitch and 
Galanter was calculated using the proportions of false alarms and hits. 
Therefore, in the work of Miller (1996) the parametric value of the variance 
is calculated, whereas in Kadlec’s study (1999) the estimators were 
obtained. 

The difference referred to in the preceding paragraph makes it 
difficult to establish common conclusions of the two studies. Moreover, 
since in most practical situations the values of the probabilities of false 
alarms and hits are unknown, the estimation of the variance of d' must be 
calculated using proportions of false alarms and hits. Therefore, when the 
methods are compared, it is more useful to make these comparisons by 
means of the estimator of the variance. 

In this paper we assess through simulation the suitability of three 
proposed methods to estimate the variance of d' and c in yes/no 
experimental paradigms (MacMillan & Creelman, 2005): method of Miller 
(1996), method of Gourevitch and Galanter (1967), and method of Dorfman 
and Alf (1968). Note that in the two above mentioned studies has not been 
evaluated the method of Dorfman and Alf (1968). Our simulation provides 
an empirical estimate of the variance of d ’, and estimates obtained by the 
three procedures. Furthermore, the estimates of the variance of d’ will be 
compared with the parametric value of variance of d’ calculated using the 
procedure of Miller (1996). The merits of the three methods are assessed by 
an evaluation of their bias and precision for a range of values of d, C, and 
N. Note that in the two above-mentioned studies has not been evaluated the 
precision. In the study presented here it is possible to evaluate the accuracy 
as both the estimated variance of d' and its parametric value will be 
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calculated. We begin providing a brief sketch of the SDT indices, the two 
main methods proposed to calculate the variance of d’ , and the three 
procedures proposed to estimate that variance. Then we describe the 
simulation and finally assess the results of the study reaching some 
conclusions and suggesting practical guidelines. 

Signal Detection Theory (SDT) indices 

There are many indices to characterize the performance in various 
contexts that can be analyzed from the SDT (MacMillan & Creelman, 
2005). Although we have a variety of parametric indices that assume 
different assumptions, and a number of nonparametric indices, we focus 
here on the two parametric indices more widely employed. The first, d, is a 
measure of sensitivity and is defined as the distance between the expected 
values of the variable of evidence for a target (signal) stimulus and a non¬ 
target (noise) stimulus, expressed in standard deviations metric. The second, 
C, is an index of the response criterion or response bias, which is defined as 
the distance between the reference value to choose the response and the 
value corresponding to the intersection between the distributions. Under the 
NH model the value of the intersection is equidistant from the expected 
values (figure 1). Put another way, it is assumed for the noise stimuli an 
approximate N(0; 1) distribution and for the signal stimuli an approximate 
distribution N (6; 1). Therefore, the sensitivity parameter, d, is the mean of 
the signal distribution. 

Suppose an experiment in which there are N s trials with a signal and 
N„ trials with noise, and the answers contain H hits and F false alarms. We 
get the hits ratio, P H = H / N s , and the false alarms ratio, P F = F / N n . The 
estimates of sensitivity, d\ and the response criterion, c, can be calculated 
from P H and P, (Macmillan & Creelman, 2005). The d’ statistic is defined 
as, 

d'= z H -z F [1] 

where z H and z F are estimates of the values of the standard normal whose 
cumulate probabilities equal the probabilities to give a yes response to a 
target stimulus and to a noise stimulus, respectively. The corresponding 
empirical proportions of hits and false alarms, P H and P F , are estimations of 
the true probabilities, jz h and jz f , as these are unknown. That is, 
z H = <P~'(P H ) and z F = 0“'(/ 3 / , ) . Likewise, c is defined as, 
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c = - 


2 h + z F ) 


[ 2 ] 


In the example of figure 1 the distance between the expected values 
equals 2 standard deviations (r) = 2) and the curves intersect at z = 1. The 
value corresponding to the response criterion stands at 0.5: half standard 
deviations to the left of the crossing value (C = -0.5). Consequently, when a 
target stimulus is presented the probability of a yes response, jz h , is 0.9332, 
whereas the probability of a yes response to a noise stimulus, jt F , is 0.3085. 


S 

I-1 



Figure 1. Example of the d and C values for a specific case (see the 
text): Jt H - 0.9332 and Jt F = 0.3085. 


Procedures to calculate the variance of d' 

What indeed we are interested in are the parameters (r) and C), but 
what we know in virtually all practical occasions are their estimators, d' and 
c. When the information is collected through a limited number of trials with 
signal and noise ( N s and N n ), the statistics do not exactly match the 
parameter values, but show some deviation due to sampling variance. 
Knowing the sampling variance allows assess the properties of an estimator. 
We can choose among several alternative estimators the one having more 
suitable properties, based on its expected value and its variance. 
Specifically, other properties being equal it is preferable an unbiased 
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estimator; one for which the expected value is the own parameter intended 
to estimate. Furthermore, other properties being equal it is also preferable 
an estimator with high precision (low variance) because in the long run its 
values tend to look more to the parameter value. 

Obtaining the variance of an estimator is not always easy or 
straightforward, as it might seem. In fact, as several proposals to calculate 
the variance of cl' have been made, it is desirable to know which one (and in 
what conditions) provides estimates closer to the actual variance. We focus 
in d’ because c is so closely related to it that the results and conclusions for 
d’ can be safely generalized to c (see equations [1] and [2]). 


Method of Gourevitch and Galanter 

One of the first attempts to develop procedures to test hypotheses 
about d and C is due to Gourevitch and Galanter (1967). They proposed an 
approach to the variance of d' assuming the NH model. Such an approach is 
obtained by developing a Taylor series of the standard normal distribution, 
considering only the first two terms of the series. With this procedure the 
following formula is reached by linear approximation, 

' (1 ~ K H ) + K F ‘ (1 ~ n F ) [ 3 ] 

N s -cp 2 (z H ) N n ■ cp 2 (z F ) 

where jz h and Jt F are, respectively, the probabilities of a hit and a false 
alarm; z H and z F are the values of the standard normal distribution 
associated, respectively, with cumulative probabilities equal to jz h and Jt F ; N s 
and N„ are the number of trials containing a signal and noise, respectively; 
and <p is the probability density function of the standard normal distribution. 


Method of Miller 

This author proposes a method for calculating the variance based on 
the exact distribution of z H and z F . Those two values are random 
variables distributed as binomials: B(N S ; Jt H ) and B (N n ; Jt F ). Taking in 
account [1] and that they are assumed independent: 

_2 


1 _2 
^d' ~ &z„ 


+ a: 


[4] 


Following Miller (1996), the variance of z H is: 


o\ = 
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[5] 
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where E(z H ) is the expected value of z H calculated by the expression: 


£(z„ ).>>-(— )• , V„-(l-a W ) 


Both in [5] and [6], <J>"‘ is the inverse of the cumulative probability of 
the standard normal. Recalling that z H ~ B(A(.; jt H ) it is obvious that the 
expression [5] is the variance of the random variable z H , since 


- ^-i 1 , i . n 

z,, = O — and -is P H . 

N N 

\ iy s / Iy s 

2 

The equation defining 07 is similar to [5], but replacing N s by N n and 

Jt H by Ji F . Likewise, the expected value of z F , E(z F ), would be obtained 
similarly to [6]. 


Procedures to obtain an estimation of the variance, a], 

The problem with the two methods above is that to calculate the 
variance of the sensitivity statistic, <jj,, with the formulae proposed by 
Gourevitch and Galanter (1967) and Miller (1996) it is necessary to know 
both ji h and Jt F . But as in most practical contexts these values are unknown, 
their estimates must be used: P H as an estimate of jz h and P F as an estimate 
of jt f . Then, what can be obtained are estimates of the variance, a],. The 
variance estimated for the d’ values is also a random variable, as it is 
calculated using the values of the variables P H and P, (as in the formulae of 
this section) instead of the constants Jt H and Jt F (as in [3], [5] and [6]). 

Three methods to estimate that variance will be evaluated in the 
simulation study presented below: the two methods already described but 
using the sample estimates instead of the parametric probabilities, and a 
maximum likelihood method (Dorfman & Alf, 1968; Kaplan, 2009). 

Method of Gourevitch and Galanter 

The estimation method based on Gourevitch and Galanter (1967) 
replaces jz h with P H and Jt F with P, in equation [3]; it reads as: 

d 2' Ph-(I-Ph) , P F -(1-P F ) 

N s '<p 2 (z H ) N n -cp 2 (z F ) 


[ 7 ] 



156 


M. Suero, et al. 


Method of Miller 

Similarly, in the procedure of estimation based on Miller’s (1996) 
method ji h and Jt F are replaced with P H and P, in equations [5] and [6] as: 


n, r 
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IN A 


K N, 


V 1 


[ 8 ] 

[9] 


~ 2 


The equation defining (J z is similar to 
and P H by P, . The same logic is applied to 
E(z f ). 


[8], but replacing N s by N„ 
obtain the expected value, 


Method of Dorfman and Alf 

The aim of the procedure proposed by these authors is to estimate the 
parameters involved. Unlike the previous two methods, instead of using the 
equations [1] and [2] for calculating the estimators they obtain the estimates 
d’ and c using the method of maximum likelihood. Adapting the logarithm 
of the l ik elihood function (equation 4 in Dorfman and Alf, 1968) for the NH 
model and keeping C constant along the trials, this function is equal to: 


LogL = F ■ log 


f-c-U 

+ (N n - F)' log 

1-0 

f-C-U] 

v 2 /. 



V 2 /_ 


H- log 


Ol --C 
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(N s -H )Tog 


6 \ 

1-01 - -C 
2 , 


[ 10 ] 


To estimate the parameters d and C they must be obtained the values 
that maximize the expression [10]. In addition, it is obtained the variance- 
covariance matrix of the estimators. In the main diagonal of this matrix can 
be found a], as an estimate of a],. Both the estimates and the variance are 
obtained by numerical methods, as for example RSCORE (Dorfman, 1982) 
or ROC FIT (Metz, 1989). 
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The problem with extreme frequencies 

In order to apply most of the equations above sometimes is necessary 
to obtain z values associated with P H and/or P, ratios equal to 1 or 0. In 
those cases the corresponding z values are ±o°, respectively, and d' is 
undefined. So, o\ cannot be calculated. Several alternatives have been 
proposed to face this problem (see Brown & White, 2005, or Hautus & Lee, 
2006, for a comparison of different methods and other alternatives), (a) The 
Log-linear correction (Snodgrass & Corwin, 1988) is applied to all 
frequencies (whatever its value); it is defined as (H + 0.5) / (N s + 1) for hits 
and (F + 0.5) / (N„ + 1) for false alarms, (b) The ±0.5 correction (Murdock 
& Ogilvie, 1968) is applied only if the frequency is 0 (being replaced by 
0.5) or N (being replaced by N - 0.5), where N is the number of signal or 
noise trials, as appropriate (alternative values to ±0.5 have been also 
proposed for the correction; see Miller, 1996). (c) Removal of the 
proportions that equal 0 and 1 (Miller, 1996). In this procedure the 
distribution of the proportions of hits and false alarms are truncated, and 
therefore the distributions of the z values associated with such proportions 
are also truncated. For example, if the procedure of Miller is applied, the 
summatories appearing in equations [8] and [9] would take values from i = 
1 to N s - 1, eliminating the addends equal to 0 (z = -oo) and 1 (z = o°). 

In sum, the conclusion from many studies has been that the ±0.5 
correction is the choice proposed for the most common situations. 
Furthermore, Miller (1996) shows that this correction and the removal 
correction have comparable performance, and better than a correction with 
a constant less than 0.5. In a Monte Carlo simulation, Hautus (1995) 
concludes that log-linear correction is better than ±0.5 correction in order 
to estimate 6. However, as Kadlec (1999) explains, the simulation 
conditions used by Hautus are not very realistic (extremely low C criteria 
and high jt H ), being necessary new simulations with more realistic 
conditions before to accept this conclusion. Moreover, the ±0.5 correction 
uses all data obtained and changing only some of them (in some situations, 
the probability of applying this is very small). In sum, our choice in this 
research is the ±0.5 correction. 

Objectives 

In previous studies (e.g., Jesteadt, 2005; Kadlec, 1999; Miller, 1996) 
several procedures to calculate the variance of d’ and c have been 
compared, but their performance has been assessed by means of the values 
provided by formulas l ik e [3] - [6]. However, the use of those formulas 
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requires knowing the parametric values Jt H and ic F . The focus in those 
studies are the estimators d’ and c, and how well the cited formulas describe 
their behavior. On the contrary, we focus here in the estimation of the 
variance itself, a ],. In real contexts both ji h and jz f are unknown. Contrary 
to those previous studies, we focus here in assessing the properties of the 
estimators of the variance when P H and P F replace jz h and Jt F . The merits of 
the three methods are assessed by an evaluation of their bias and precision 
for a range of values of d, C, and N. 


METHOD 


Statistical model 

It was assumed the SDT-NH model, with mean 0 and variance 1 for 
the noise trials and with mean d (the sensitivity parameter) and variance 1 
for the signal trials. Both the frequencies of hits, H, and false alarms, F, 
were obtained by generating random values. To do that we defined signal 
and noise distributions, as also the sensitivity parameter, 6. In addition, we 
set several values for the criterion, C, and the number of signal and noise 
trials, N s and N„. From these values, the probabilities of hits (jt H ) and false 
alarms (jt f ) were calculated. Once determined the jz f and N„ values for a 
given condition, the frequency of false alarms, F, follows a binomial 
distribution B (N„; jz f ) [the frequency of hits, H, follows a binomial 
distribution B(A(; ji h )\. So, the frequencies of hits and false alarms were 
obtained in the simulation as random values from these distributions. 

Conditions of the simulation 

Three variables were manipulated: the number of trials of each type, 
N s and N„; the sensitivity, d; and the criterion, C. With respect to the number 
of trials, both signal and noise always had the same amount: 20, 30, 50, or 
80 trials. For d, the following values were considered: 0.5, 1, 1.5, 2, 2.5, or 
3. The criterion values, C, were -0.5, 0, or 0.5. Table 1 shows the values of 
Jtp and 3% corresponding to each pair of values of d and C. Given the 
combinations of the levels of the three manipulated variables 72 conditions 
were simulated (4 numbers of trials x 6 sensitivities x 3 criteria). They were 
obtained 100,000 repetitions (ie, 100,000 pairs of frequencies of hits and 
false alarms) for each simulated condition. A program written by the 
authors in R (R Core Team, 2015) performed the simulation. 
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Table 1. Values of Jt F and jt h used in the simulations. They have been 
calculated from the values of b and C, assuming the NH model (see the 
text). 




C 

JTf 

itn 

6 

= 0.5 

-0.5 

0.59871 

0.77337 



0 

0.40129 

0.59871 



0.5 

0.22663 

0.40129 

& 

= 1.0 

-0.5 

0.50000 

0.84134 



0 

0.30854 

0.69146 



0.5 

0.15866 

0.50000 

6 

= 1.5 

-0.5 

0.40129 

0.89435 



0 

0.22663 

0.77337 



0.5 

0.10565 

0.59871 

6 

= 2.0 

-0.5 

0.30854 

0.93319 



0 

0.15866 

0.84134 



0.5 

0.06681 

0.69146 

5 

= 2.5 

-0.5 

0.22663 

0.95994 



0 

0.10565 

0.89435 



0.5 

0.04006 

0.77337 

6 

= 3.0 

-0.5 

0.15866 

0.97725 



0 

0.06681 

0.93319 



0.5 

0.02275 

0.84134 
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Data analysis 

For the pair of frequencies of hits and false alarms of each repetition 
(H and F) we calculated d' and c assuming the NH model, using equations 
[1] and [2]. In the event that the frequencies were equal to zero or to the 
number of trials (N) the ±0.5 correction was applied (Murdock & Ogilvie, 
1968): if the frequency is 0 it is replaced by 0.5 and if the frequency is equal 
to the number of trials it is replaced by (N s - 0.5) or (N n - 0.5). Thus, within 
each simulated condition 100,000 values of d' and c were obtained. Then, 
we calculated for each condition the mean and variance of those 100,000 
values of d' and c. We checked for departures of those means and variances 
from the population values. Of course, the population values for the means 
are the d values used to generate the data. The population values of 
variances are those provided by Miller’s, formula [5]. We also calculated 
the population values of the variances of d' by Gourevitch and Galanter’s, 
formula [3]. 

Table 2 allows assessing the process of data generation by comparing 
the population value with the means of the d’ values. The discrepancies 
observed in the tables are mainly due to the application of the correction 
due to zero and N frequencies. Table 3 allows assessing the process of data 
generation by comparing the population values with the variances of the d’ 
values. It must be remembered that while Miller’ method is an exact 
calculation, Gourevitch and Galanter’s method is only an approximation. 
The discrepancies observed in the variance provided by Miller’s formula 
are mainly due to the application of the correction due to zero and N 
frequencies. Furthermore, and as was expected, the discrepancy observed in 
the population value of the variance of d’ provided by Gourevitch and 
Galanter’s method are greater than those obtained by Miller’s formula, and 
these depend on d and the number of trials. 

Hereinafter, the estimates of the variance of d' obtained with the three 
methods set forth in the introduction are compared with the population 
value obtained with the Miller’s method. 
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Table 2. Means of the 100,000 empirical estimates of d’ obtained for 
each simulated condition. 


N 



C 

20 

30 

50 

80 

<5 = 0.5 

-0.5 

0.53151 

0.51999 

0.51151 

0.50659 


0 

0.52364 

0.51229 

0.50859 

0.50573 


0.5 

0.52913 

0.52040 

0.51082 

0.50675 

<5 = 1.0 

-0.5 

1.06337 

1.04487 

1.02485 

1.01522 


0 

1.04870 

1.02963 

1.01682 

1.01105 


0.5 

1.06171 

1.04557 

1.02662 

1.01660 

<5 = 1.5 

-0.5 

1.57903 

1.57085 

1.54670 

1.52720 


0 

1.58663 

1.55545 

1.53007 

1.51921 


0.5 

1.57902 

1.57161 

1.54502 

1.52743 

i 

o 

<N 

II 

•o 

-0.5 

2.06265 

2.08204 

2.07389 

2.04553 


0 

2.12206 

2.08785 

2.05314 

2.02962 


0.5 

2.05950 

2.08409 

2.06884 

2.04591 

<5 = 2.5 

-0.5 

2.49394 

2.55297 

2.58188 

2.57152 


0 

2.63430 

2.62635 

2.58437 

2.54894 


0.5 

2.49282 

2.55496 

2.58047 

2.56872 

<5 = 3.0 

-0.5 

2.87611 

2.96770 

3.04586 

3.07399 


0 

3.07068 

3.13005 

3.12346 

3.08141 


0.5 

2.87627 

2.97010 

3.04313 

3.07363 
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Table 3. Variances of the 100,000 empirical estimates of d’ obtained for 
each simulated condition ( Emp ), variances calculated with Miller’s 
exact method (M), and Gourevitch and Galanter’s method (G&G). 


N 



C 


20 

30 

50 

80 

d 

II 

‘O 

-0.5 

Emp 

M 

G&G 

0.20487 

0.20557 

0.17698 

0.13093 

0.13108 

0.11799 

0.07518 

0.07503 

0.07079 

0.0458 

0.0458 

0.04425 


0 

Emp 

M 

G&G 

0.17848 

0.17775 

0.16069 

0.11419 

0.11414 

0.10713 

0.06681 

0.06666 

0.06428 

0.04076 

0.04108 

0.04017 


0.5 

Emp 

M 

G&G 

0.20607 

0.20557 

0.17698 

0.13168 

0.13108 

0.11799 

0.07399 

0.07503 

0.07079 

0.04566 

0.0458 

0.04425 

<5 = 1 

-0.5 

Emp 

M 

G&G 

0.22101 

0.22057 

0.19253 

0.14809 

0.14834 

0.12835 

0.08447 

0.08439 

0.07701 

0.05081 

0.05072 

0.04813 


0 

Emp 

M 

G&G 

0.19695 

0.19799 

0.17212 

0.12426 

0.12528 

0.11475 

0.07189 

0.0723 

0.06885 

0.04422 

0.04432 

0.04303 


0.5 

Emp 

M 

G&G 

0.22182 

0.22057 

0.19253 

0.14834 

0.14834 

0.12835 

0.08444 

0.08439 

0.07701 

0.05076 

0.05072 

0.04813 

<5=1.5 

-0.5 

Emp 

M 

G&G 

0.22746 

0.22538 

0.22196 

0.16987 

0.16941 

0.14798 

0.10428 

0.10318 

0.08879 

0.06113 

0.06101 

0.05549 


0 

Emp 

M 

G&G 

0.23475 

0.23338 

0.19327 

0.14838 

0.14803 

0.12885 

0.08353 

0.0834 

0.07731 

0.05043 

0.05052 

0.04832 


0.5 

Emp 

M 

G&G 

0.22544 

0.22538 

0.22196 

0.16868 

0.16941 

0.14798 

0.10335 

0.10318 

0.08879 

0.06084 

0.06101 

0.05549 

<5 = 2 

-0.5 

Emp 

M 

G&G 

0.21204 

0.21279 

0.27189 

0.1759 

0.1764 

0.18126 

0.12645 

0.12685 

0.10875 

0.0806 

0.08034 

0.06797 


0 

Emp 

M 

G&G 

0.27011 

0.26952 

0.22798 

0.18503 

0.18589 

0.15199 

0.10414 

0.10386 

0.09119 

0.06127 

0.06136 

0.057 


0.5 

Emp 

M 

G&G 

0.21383 

0.21279 

0.27189 

0.17605 

0.1764 

0.18126 

0.12727 

0.12685 

0.10875 

0.07967 

0.08034 

0.06797 

<5 = 2.5 

-0.5 

Emp 

M 

G&G 

0.19592 

0.19515 

0.35494 

0.16502 

0.16477 

0.23662 

0.13572 

0.13621 

0.14197 

0.10347 

0.10388 

0.08873 


0 

Emp 

M 

G&G 

0.27204 

0.27301 

0.28323 

0.22346 

0.22469 

0.18882 

0.13967 

0.1397 

0.11329 

0.0808 

0.08094 

0.07081 


0.5 

Emp 

M 

G&G 

0.1974 

0.19515 

0.35494 

0.1638 

0.16477 

0.23662 

0.13625 

0.13621 

0.14197 

0.10354 

0.10388 

0.08873 
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Table 3 (continued). Variances of the 100,000 empirical estimates of d’ 
obtained for each simulated condition ( Emp ), variances calculated with 
Miller’s exact method (Af), and Gourevitch and Galanter’s method 
(G&G). 



Emp 

0.18148 

0.15244 

0.12565 

0.10944 

-0.5 

M 

0.18147 

0.15158 

0.1255 

0.11043 


G&G 

0.49534 

0.33022 

0.19813 

0.12383 


Emp 

0.22549 

0.22835 

0.18136 

0.11696 

0 

M 

0.22759 

0.22752 

0.1814 

0.11636 


G&G 

0.37165 

0.24777 

0.14866 

0.09291 


Emp 

0.18157 

0.15182 

0.12542 

0.11103 

0.5 

M 

0.18147 

0.15158 

0.1255 

0.11043 


G&G 

0.49534 

0.33022 

0.19813 

0.12383 


Within each condition and for each pair of H and F values we 
obtained estimates of the variance of d' by the three methods set forth in the 
introduction: 

(a) Method of Gourevitch and Galanter (1967). Equation [7] was employed 
for each pair of proportions (P H and P F ). Thus, for each condition we 

obtained 100,000 estimates of the variance (100,000 values of O'^xciG))- The 
mean and the variance of those estimates was calculated for each condition: 

( y '.. a f i (I .S . . 

d (GG) °ixGG) 

(b) Method of Miller (1996). We repeated the process of the above method 
but with the equations [8] and [9] (and their counterparts for false alarms), 

also obtaining 100,000 variance estimates (100,000 values of tjJ. (M) ). 
Finally, the mean and the variance of the estimates was calculated: 

&d\M) and S ll XM) - 

(c) Method of Dorfman and Alf (1968). Equation [10] was used as 
likelihood function for this case, obtaining 100,000 variance estimates 

(100,000 values of £jj, (D/4) ) and then calculating the mean and the variance: 

&d\DA) and S al {DA) ■ 

Programs in R (R Core Team, 2015) developed by the authors were 
used for the calculation of d' and c, as well as (J) YCC) , (jj, (V/) and (see 
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the appendix). The bbmle library (Bolker, 2015) was used for the process of 
maximum likelihood estimation. 

Assessing the performance of the methods of estimation 

The bias of the three estimates was assessed by calculating the 
discrepancy between the population values and the means of the empirical 
estimates. The bias of an estimate is defined as the difference between the 
expected value of the estimate and the parameter: bias = E(Q) - 0. 
However, as the importance of the amount of bias must be assessed in 
relative terms we will calculate the relative bias of the three estimates 
(Burton, Altman, Royston & Holder, 2006), expressed as a percentage, 

Fff) ) _ f) 

Relative bias =-—-• 100 [11] 

0 

A j 

where E(Q t ) is the mean estimates of cr d , computed with each of the three 

methods (d 'j, (GG) , and <J dWA) ) and 0 is the variance of d' obtained 

with Miller’s exact method. A discrepancy close to zero would indicate that 
the method of estimation is accurate, while positive and negative 
differences would reflect, respectively, over- and underestimates. 

Although the amount of bias must be the main criterion to compare 
several methods of estimation, it must be complemented with a measure of 
precision. An unbiased estimate that has a very large variance could be 
assessed as worse than an estimate with small bias but with a much smaller 
variance. A good estimate must involve a balanced combination of accuracy 
and precision. To do that we calculate the mean squared error: 
MSE = is(0* - 0) 2 ; it can be expressed as a function of the bias and the 
variance of the estimator, 

MSE = bias 2 + Var(@ t ) [12] 

In any practical situation, the researcher has a single estimate of the 
parameter. Therefore, it is reasonable that the criterion for choosing an 
estimator be the (squared) expected difference between the estimate and the 
parameter. This is done by mean of the MSE. When comparing the MSE 
values of two competing estimators the amount of bias is penalized by 
larger variances. 
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RESULTS AND DISCUSSION 

Our main interest is on the ability of the three methods to estimate the 
actual variance when only the estimates of the probabilities (P H and P F ) are 
known. The results for the relative bias are presented in figure 2. This has 
several striking aspects. First, for all the conditions simulated the relative 

bias of an< ^ &d\DA) are virtually identical. In fact, at a first glance the 

differences are not obvious in the figure because their functions are literally 
over imposed. Our first conclusion is that, at least for the conditions 
simulated here, the expected values of both variances are indistinguishable 

for practical purposes: tr], (GG) = 0 }- {DA] . Second, in general for the conditions 
simulated the variance with less relative bias is that obtained with Miller’s 
method the average of the estimates obtained by this method is 

closer to the true value than the average obtained by the other two. The 
larger of these discrepancies is 17.9% (condition with d = 3, C = 0, N s = N„ 
= 30). Third, the magnitude of the bias with the method of Miller does not 
change very much across the conditions, and the fluctuations do not show 
any obvious pattern (they are not systematically associate to d, C, or N s and 
N„). Fourth, in some conditions the G&G and D&A methods overestimate 
considerably the variance in the long run. Those discrepancies increase the 
higher is d, the smaller are N s and N„, and the farther to 0 is C. In some 
conditions the relative bias exceeds 140% (for example, the relative bias of 
the variances estimated by these two methods is 140.5% in the conditions 
with d = 3, C ^ 0, and N s = N„ = 30). 

However, there is a number of conditions for which the amount of 
bias is not larger for the G&G and D&A methods than for the Miller’s 
method. See, for example, the conditions with d < 1, or the conditions with 
N = 50 or 80, with C = 0, no matter the value of d. That is why sometimes 
has been concluded that there is a range of conditions where those two 
methods are a reasonable alternative to the Miller’s method. 

However, a good estimate must have small (if any) bias and large 
precision (small variance). The MSE reflect some balance between both 
criteria. The results for the MSE are presented in figure 3 and table 4. 
Several aspects must be highlighted also on it. First, the G&G and DA 
methods are again practically indistinguishable. Second, the MSE for 
Miller’s method outperforms the other two along a range of the conditions 
simulated, with a few exceptions. In those exceptional occasions (in bold in 
table 4) the larger MSE value for Miller’s method is as small as 0.00042 
(condition with d = 2,C = 0,N s = N n = 50). 
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Figure 2. Relative bias (expressed as a percentage) of the three 
estimation procedures of a], (GG: Gourevitch & Galanter; M: Miller; 
DA: Dorfman & Alf). 
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Figure 3. MSE of the three estimation procedures of o 2 d , (GG: 
Gourevitch & Galanter; M: Miller; DA: Dorfman & Alf). 
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Table 4. Mean squared error of the estimates of the variance of d’ 
obtained with the three methods for each simulated condition. 





CN 


CN 

_ 

i—i 



•n 

OO 

8 

CO 

© 

CN 

N" 

CN 

o 


o 

O 


© 

i-H 

© 

i—i 

VO 

© 


CN 

i—i 

© 

CN 

ON 

8 

o 

8 

8 

o 


8 

8 

8 

8 

8 

© 

© 

CN 

© 

8 

CN 

© 

VO 

© 

© 

vn 

© 

O 


o 

O 


© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 


O' 

o 


o 

o 

© 

© 

d 

d 

© 

© 

© 

© 

o 

© 

d 







ITi 

CN 

if) 

_ 

ON 



CO 

VO 




o 


o 

o 


© 


© 


CN 

© 

CN 

CN 

co 

CN 

N- 

•vf 


8 

o 

8 

O 

o 

o 

© 

© 

© 

© 

© 

© 

© 

© 

8 

© 

© 

8 

8 

© 

© 

8 

8 

8 

8 

o 


O 

o 


© 

© 

© 

© 

© 

© 

o 

o 

© 

O 

o 

© 

© 

© 


o 

o 


© 

© 

© 

© 

o 

© 

o 

o 

© 

o 

o 

o 

© 



_ 

<N 


CN 

_ 

_ 



«n 

oo 

On 

co 

© 

CN 

N" 

CN 

8 


8 

8 


O' 

i—i 

© 


VO 

8 

vn 

© 

N 

1 1 

o 

CN 

ON 



© 

© 

© 

© 

© 

© 

CN 

© 

CN 

VO 



o 

o 

O 

O 

o 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

o 


o 

O 


© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 


o 

o 


© 

o 

o 

d 

d 

d 

o 

o 

o 

o 

o 

d 

d 



co 


CN 

VO 

CO 

r- 

© 

© 

CN 

r- 

VO 

r- 

VO 

N- 

CN 

CO 

o 


© 


o 

i—i 

r- 

© 

r- 

CO 

CO 

CN 

CN 

N- 

c. 

CN 

vn 

i—« 

o 

o 

o 

8 

o 

o 

8 

© 

© 

© 

© 

© 

© 

8 

CN 

© 

8 

CN 

© 

© 

© 

vn 

© 

VO 

8 

VO 

o 


O 

o 

O 

O 

o 

© 

O 

© 

© 

o 

© 

o 

o 

o 

o 

© 

© 


o 

O' 

o 

© 

© 

© 

o 

o 

o 

© 

© 

o 

o 

o 

o 

d 

00 


00 

O 

TT 

© 

N- 

»ft 

N" 

© 

CN 

CN 


ON 

r- 

© 


ON 

o 

o 

o 

CN 

© 

CN 

N 


cl 

CO 

8 

CO 

VO 

8 

VO 


00 

VO 

o 

© 

o 

® 

O 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

o 

o 

o 

O 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

o 

o 

o 

O 

© 

© 

o 

© 

© 

o 

© 

© 

© 

© 

© 

© 

o 

© 

o 

o 

® 

® 

© 

© 

o 

© 

d 

© 

© 

d 

d 

d 

© 

© 

d 

© 

N- 


co 

«ri 

r 1 

VO 

co 

r- 

© 

© 

CN 

r- 

VO 

r- 

VO 

N" 

CN 

CO 

o 


8 

i—• 

8 

i—t 

r- 

8 

r- 

CO 

CO 

CN 

CN 

N- 

On 

CN 

vn 

l—l 

o 


O 

© 

© 

© 

CN 

© 

CN 

«n 


«n 

VO 


VO 

o 

o 

o 

o 

8 

O 

8 

o 

8 

8 

o 

o 

o 

o 

8 

i-^ 

o 

l—l 

o 


o 

o 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 


© 

© 

d 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

CN 


CN 

ON 


© 

oo 

CO 

r- 

© 

ON 


«n 


r- 

CN 

CN 

On 

co 

8 

co 

o 


T—< 

ON 

VO 

On 

1—1 

i-^ 

1—1 

N- 

OO 

r- 

OO 

On 

© 

o 

o 


o 


CN 

© 

CN 

r- 

CN 

r- 

OO 


oo 

oo 

r- 

ON 

o 

o 

o 

o 

o 

© 

© 

© 

© 

© 

© 

© 

1-H 

© 

y—l 

CO 

i—i 

CO 

o 

o 

o 

© 

© 

o 

O 

© 

O 

© 

o 

© 

© 

o 

© 

© 

© 

© 

© 

d 

o 

o 

d 

d 

d 

d 

d 

d 

o 

d 

d 

d 

© 

o 

© 

© 


r- 

o 

oo 

00 

oo 

oo 


«/■) 

CO 



CN 

«n 

CN 

VO 

00 

VO 

co 

o 

co 

<N 

CN 

CN 

8 

•n 

8 

ON 

vn 

ON 

© 

CN 

© 

On 

r- 

ON 

© 

o 

© 

o 

© 

O 

© 

o 

© 

o 

l-H 

i-^ 

i—i 

o 

CN 

© 

o 

o 

o 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

o 

© 

© 

o 

o 

o 

o 

© 

o 

© 

© 

o 

© 

© 

© 

© 

o 

o 

© 

© 

© 

d 

o 

© 

o 

© 

© 

o 

© 

d 

© 

© 

© 

© 

© 

o 

© 

© 

© 

CN 


CN 

On 


© 

oo 

CO 

r- 

© 

ON 

_ 


l-^ 

r- 

CN 

CN 

ON 

co 

8 

ro 

O 

y—> 

y—i 

On 

VO 

ON 

i—i 

i—i 

1—1 


oo 

r- 

OO 

On 

© 

© 

o 


© 

l—l 

CN 

© 

CN 


CN 

r- 

oo 

vo 

oo 

oo 

r- 


8 

o 

o 

o 

o 

8 

8 

© 

o 

8 

© 

o 

8 

8 

8 

8 

o 

8 

o 

CO 

o 

© 

co 

o 

© 

o 

e 

© 

d 

d 

O 

o 

o 

O 

© 

O 

d 

O 

O 

o 

© 

© 

CN 

r- 

oo 

CN 

co 

VO 


ON 

oo 

r- 

*n 

r* 

tj- 

r- 

ON 

© 


«n 

N- 

CN 

N- 

CN 

00 

CN 

© 

r- 

l-H 

CN 

8 

00 

i-^ 

oo 

N- 

CN 

in 

oo 


O 


CO 

o 

co 

r- 

CN 

r- 

OO 

r- 

© 

r- 

On 

CO 

CN 

CO 

O 

o 

o 

o 

o 

© 

© 

© 

© 

1—1 

© 

i—i 


i—i 

co 

r- 

VO 

r- 

o 

o 

o 

o 

o 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

d 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 




r- 

r- 

ON 

ON 

CN 

co 

© 

VO 

CO 

On 

On 

OO 

r- 

© 

oo 

s 

ro 

s 

<n 

vn 

«n 

CN 

VO 

CN 

N- 

r- 

N- 

CN 


CN 

i—i 

ON 

l—l 

o 

o 

o 

© 


© 



i—i 



co 


i—i 

CN 

l—l 

o 

o 

o 

o 

o 

o 


8 

© 

© 

8 

© 

o 

© 

o 

© 

o 

8 

© 

O 

© 

O 

© 

o 

8 

8 

8 

© 

© 

© 

o 

© 

© 

O 

© 

© 

© 

© 

o 

© 

o' 

o 

o 

© 

O 

o 

© 

CN 


oo 

CN 

ro 

VO 


On 

oo 

r- 

»n 

r- 

N- 


Os 

© 


<n 


CN 


CN 

oo 

CN 

© 

r- 

l—l 

CN 

8 

00 


oo 

N- 

On 

vn 

oo 

i—i 

o 

*—t 

co 

o 

CO 

i- 

CN 

r- 

oo 

r- 

© 

r- 

On 

CO 

CN 

CO 

8 

8 

8 

5 

8 

8 

8 

8 

8 

O 

8 

i—i 

o 

8 

i—i 
© 

CO 

© 

r- 

o 

VO 

© 

© 

© 

© 

© 

© 

d 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

© 

»n 

© 

1 

o 

| 0.5 

»n 

© 

i 

o 

0.5 

in 

d 

© 

| 0.5 

in 

© 

© 

0.5 

in 

d 

© 

| 0.5 

in 

© 

i 

© 

0.5 


«n 






«n 



CN 



»n 



CO 



© 






■ i 





CN 





II 



II 



ii 



II 



ii 



II 



^5 


























Variance ofSDT indices 


169 


Practical implications for meta-analysis 

As we noted in the introduction, a meta-analyst usually obtains 
estimates of effect sizes by a weighted combination of independent 
estimates of that effect size. The most common weighting scheme is that 
based on the reciprocal of their variances. When a study reports the mean 
and variance of the values of d’ in two samples of participants the meta¬ 
analyst has enough information for applying those procedures. For example, 
in a study by Rhodes and Jacoby (2007) there are conditions with 
“frequent” and “infrequent” targets. They report the means and standard 
deviations of the d’ values in the samples. In those cases the sample 
variance S 2 d , can be employed as an estimate of a ],. However, many papers 
only report the statistics associated with hits and false alarms rates, and 
sometimes the values of d' and c associated with the average rates of hits 
and false alarms. That information could not be enough to obtain the desired 
estimate of the variance. In this second group of studies a], must be 
estimated with procedures such as those assessed here. Our results allow us 
to assess the different alternatives in terms of bias and precision. Many 
meta-analyses that have been made from the rates of hits and false alarms 
could be re-done with the statistics d' and c, but this requires to have 
formulas to calculate estimates of d and a], from the means and variances 
of the hits and false alarms rates. 

Another problem for the meta-analyst is that the procedures studied 
here are suitable only if the assumption that all participants in an 
experimental condition share the same parameter values (d and C) holds. 
However, in many situations it is more realistic to assume that there are 
individual differences in sensitivity and / or criteria among participants of 
the same experimental condition. To cover this possibility these formulas 
must be adapted to those situations. We are already working on these new 
developments (Suero, Botella, & Privado, in preparation). 

Among our medium term goals is to develop procedures for meta¬ 
analysis of studies within a SDT framework that report partial information. 
It is very frequent that the studies in several topics only report statistics 
associated with the rates of hits and false alarms. Consequently, the basis 
for those meta-analyses are those statistics (e.g., Gardiner, Ramponi, & 
Richardson-Klavehn, 2002, in recognition memory; Heinrichs & Zakzanis, 
1998, in sustained attention). In short, we believe that it is possible to rescue 
those studies for a meta-analysis based on d’ and c statistics and that 
acknowledges the existence of individual differences in sensitivity and / or 
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criteria. In that way, we will be able of doing better syntheses of the 
evidence in topics where SDT is a common framework. 

Conclusions 

The main conclusion of this study is that among the procedures 
compared that of Miller (1996) is the most recommended to estimate the 
variance of d’. In some previous studies it is concluded that in some 
situations, the G&G method is at least equally good, but they are based on 
the parametric variance (jt F and Jt H instead of P F and P H ) or the methods are 
assessed only according to their bias. We believe that the methods must be 
compared assessing both the bias and the MSE. When a researcher needs an 
estimate of the variance of d’ what has available are usually P F and P H . A 
good criterion is choosing the estimator for which it is expected a smaller 
(squared) difference with the population variance: the MSE. When MSE is 
taking in account, the recommended estimator must also be Miller’s method 
calculated with the sample proportions. This conclusion is valid for the 
complete range of conditions assessed in the present study (d until 3; C 
between -0.5 and 0.5; N s and N„ until 80). 

All the developments and analyses in this paper refer to data obtained 
with a Yes/No paradigm. However, our preference for Miller’s method 
converge with the conclusions of simulation studies with rating paradigms 
(e.g., MacMillan, Rotello, & Miller, 2004). The results of rating 
experiments allow generating complete ROC curves based in several points 
in the ROC space. Despite this fundamental difference, the method 
preferred is the same. 

With respect to the variance of the index of response bias, c, as it is 
based on the same information as d' and this is analyzed in a similar way, 
the conclusion regarding the estimation methods is the same. 


RESUMEN 

Metodos para estimar la varianza de algunos indices de la teoria de la 
deteccion de senales: Un estudio de simulacion. Se presenta un estudio de 
simulacion para evaluar y comparar tres metodos de estimacion de la 
varianza de las estimaciones de los parametros (3 y C de la teoria de la 
deteccion de senales (TDS). Se han propuesto varios metodos para calcular 
la varianza de sus estimadores, d’ y c. Dichos metodos han sido evaluados 
sobre todo comparando las medias y varianzas empiricas en estudios de 
simulacion en los que los calculos se han hecho con los valores parametricos 
de las probabilidades de emitir una respuesta ‘si" en un ensayo-senal 
(aciertos) y en un ensayo-ruido (falsas alarmas). En contextos practicos la 



Variance ofSDT indices 


171 


varianza tiene que ser estimada a partir de las estimaciones de esas 
probabilidades (tasas empfricas de aciertos y falsas alarmas). Los tres 
metodos para estimar la varianza comparados en la presente simulacion son 
los basados en la distribucion binomial de Miller, en la aproximacion a la 
normal de Gourevitch y Galanter y el de maxima verosimilitud propuesto 
por Dorfman y Alf. Estos se comparan en terminos de su sesgo relativo 
(exactitud) y en el error cuadratico medio (precision). Los resultados 
muestran que los dos ultimos metodos se comportan de forma indistinguible 
a efectos practicos y producen importantes errores de sobre-estimacion en 
un abanico de situaciones que sin ser las mas comunes son bastante realistas 
en diversos contextos practicos. Por el contrario, el metodo de Miller 
proporciona mejores resultados (o al menos similares) en todas las 
condiciones estudiadas. Es el metodo recomendado para obtener 
estimaciones de las varianzas de estos estadfsticos en situaciones aplicadas. 
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APPENDIX 

metatds . A R function for computing variance of d’ and other indices 
following three different methods 

# COMPUTE: 

#Variance of d' following Gourevitch & Galanter (1967). 

#Mean and variance of d’ following Miller (1996). 

#Variance of d’ and more (see OUTPUT) following MLE, Dorfman & 
Alf (1968). 

# NEEDS PACKAGE: bbmle and stats4. 

# ARGUMENTS: 

# nr number of noise trials. 

# ns number of signal trials. 

# pi_fa probability offalse alarms or its estimation false alarms rate. 

# pi_a probability of hits or its estimation proportions of hits rate. 

#OUTPUT is a list with: 

# VAR_GG variance d' Gourevitch & Galanter (1967) 

# Miller a list with: 

# Varianza variance d’ Miller (1996). 

# Val_Esp expected value d' Miller (1996). 

# ML a list with: 

#resumen fitting summary 

#p_estim a vector with d’ and c estimation 

#loglike is loglike 

#var_covar variance-covariance matrix, var_covar[l,l] is variance 
d’ 


# Correction extreme values: ±0.5 methods. Future version will include 
other methods. 
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############################################################# 

library(stats4) 

library(bbmle) 


metatds <- function(nr = 100, ns = 100, pi_fa = 0.50, pi_a = 0.50) 

{ 

#Variance d’ Gourevitch & Galanter (1967) 

var_gg <- ((pi_fa*(l-pi_fa))/(nr*dnonn(qnorm(pi_fa)) A 2)) + 
((pi_a*(1 -pi_a))/(ns*dnonn(qnorm(pi_a)) A 2)) 


#Variance d’ and Expected Value d’ Miller (1996) 

fre_fa<- c(0.5,(l:(nr-l)), nr-0.5) 
fre_a <- c(0.5,(l:(ns-l)), ns-0.5) 

prop_fa <- fre_fa/nr 
prop_a <- fre_a/ns 

z_fa <- qnorm(prop_fa, mean = 0, sd = 1) 
z_a <- qnonn(prop_a, mean = 0, sd = 1) 

prob fa <- dbinom(0:nr,nr,pi_fa) 
prob a <- dbinom(0:ns,ns,pi_a) 

v esp zfa <- sum(z_fa*prob_fa) 
v esp za <- sum(z_a Hs prob_a) 
y csp miller <- v esp za - v esp zfa 

var_zfa <- sum(((z_fa*z_fa)*prob_fa))-(v_esp_zfa*v_esp_zfa) 
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var_za <- sum(((z_a*z_a)*prob_a))-(v_esp_za*v_esp_za) 
var miller <- var za + var zfa 


mestim <- list(Val_Esp = v_esp_miller,Varianza=var_miller) 


#Variance of d’ and more (see OUTPUT) following MLE, Dorfman & Alf 
(1968). 

#It is possible other models like nonnal heteroscedastic or non-nonnal if LL 
is changed. 


LL <- function(xc, dp, mr, ms, fal, fac) 

-sum(((mr-fal)*pnorm(xc,log.p = TRUE))+(fal*pnorm(xc,lower.tail = 
FALSE,log.p = TRUE))+ 

((ms-fac)*pnorm(xc-dp,log.p = TRUE))+(fac*pnorm(xc-dp,lower.tail = 
FALSE,log.p = TRUE))) 


xc_in= qnorm(pi_fa, mean = 0, sd = 1, lower.tail = FALSE) 

dp_in= qnorm(pi_a, mean = 0, sd = 1, lower.tail = TRUE) - qnorm(pi_fa, 
mean = 0, sd = 1, lower.tail = TRUE) 


fit <- mle2(LL,start=list(xc = xc in, dp = dp in) , fixed = list(mr = nr,ms = 
ns, fal= nr*pi_fa, fac = ns*pi_a)) 

ml <- list(resumen=summary(fit), p_estim=coef(fit, exclude.fixed = 
TRUE),loglike=logLik(fit),var_covar=vcov(fit)) 


esti <- list(Var_GG =var_gg, Miller=mestim, ML=ml) 


return(esti) 

} 
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